1. Field of the Invention
This invention relates to video information retrieval.
2. Description of the Prior Art
Video images are a useful resource for entertainment and for dissemination of information. Digital video images are also increasingly being used in a wide range of multimedia applications.
The sheer volume of video information currently available to the user is overwhelming with the existence of many video libraries and archives each of which potentially stores millions of images. These video archives have a broad spectrum of users running different applications and requiring a range of services from provision of subject-specific video clips for editing purposes to video on demand. In practical terms the video archive environment must allow users to run custom applications which utilise a common database of video images and provide descriptive data related to the video images to allow the user to make an informed choice of which media file to download. The generic term for the descriptive data associated with video images is metadata.
Computer database management systems have proved to be very effective for organising text and numeric data. The most widespread database management systems are known as “relational” databases. These systems collect data and organise it as a set of formally described tables from which data can be accessed selectively and reassembled in a variety of ways without having to reorganise the data tables. The standard user and application program interface (API) to a relational database is the structured query language (SQL) which can be used for simple interactive queries as well as for more extensive data gathering for use in compiling reports.
A further example of an information management system is a web search engine. The web search engine is ideally suited for use in a multimedia environment and has three basic components:    A program known as a “spider” that goes to every page or representative pages on every web site that wants to be searchable and reads it, using hypertext links on each page to discover and read a site's other pages.    A program that creates a master index from the pages that have been read.    A program that receives a user's text-based search request, compares it to the entries in the master index, and returns results to the user.
Video archives are of very limited value to the user unless there is an information management system for images capable of delivering images based on their specific content. This video information management system is likely to require features used in database management systems as well as some of the functionality of the web search engine. One difficulty is that image and video data require a much higher bandwidth than text-based information. Downloading a video clip across a computer network can be very time consuming because of the large quantity of data involved. In some cases the user may have to download and view several video clips in real time in order to find a clip with the required information content. Thus it is very important to provide the user with adequate information about images in the archives prior to any download to increase the likelihood of the downloaded images meeting the user-specific requirements. Some users may be looking for video clips that can be used to illustrate a particular feature or issue, for example, video segments showing a particular politician or dignitary. Other users might be searching for complete programmes and news items related to a specific topic such as global warming. It would also be advantageous to the user to have unrestricted access to as many video archives as possible via a single video-specific search query.
A typical prior-art video information retrieval system for use on the world-wide web is illustrated in FIG. 1. Video source material 10 is input as raw video information 15 to an encoding and content-analysis module 20. The source material could be a digital or analogue video-cassette, an electronically stored digital video file or a broadcast signal fed directly via satellite The encoding and content-analysis module 20 takes the video source material and produces digital copies it in various alternative formats ranging from low bit-rate versions suitable for use on Internet browser plug-ins such as RealVideo™ to high bit-rate broadcast quality MPEG2 images.
On input to the video archive system the analogue or digital source material is subject to an automated content-analysis process. This typically involves the use of local intensity histograms, edge histograms, geometrical shape analysis, face detection and on-screen text extraction to establish and log the content of each image. The associated audio samples may be processed for content using speech detection algorithms. Proprietary content-analysis software such as Virage's Videologger™ has been be used for this purpose. The result is a video index 25 which summarises the content of the video material.
A video application server 30 stores the video index 25 in an appropriate format so that it is accessible to a web server 40. In addition the video application server 30 provides a flexible template system, handles client-queries and provides administration tools. Clients 60 running Internet browsers have access to the video index via the web server 40. The clients may enter search terms in a standard web search engine which is interfaced the video index so that video material can be selectively retrieved on the basis of its logged content.
The encoding and content-analysis module 20 outputs the digital video information 65 across a distribution network. The digital video information 65 is available for download to the clients via a video server 50. The video index 25 is used to search for and retrieve particular video clips required by users.